Optimal learning via local entropies and sample compression

Under margin assumptions, we prove several risk bounds, represented via the distribution depen-dent local entropies of the classes or the sizes of specific sample compression schemes. In somecases, our guarantees are optimal up to constant factors for families of classes. We discuss limi-tations of our approach and give several applications. In particular, we provide a new tight PACbound for the hard-margin SVM, an extended analysis of certain empirical risk minimizers underlog-concave distributions, a new variant of an online to batch conversion, and distribution depen-dent localized bounds in the aggregation framework. As a part of our results, we give a new upperbound for the uniform deviations under Bernstein assumptions, which may be of independent in-terest. The proofs for the sample compression schemes are based on the moment method combinedwith the analysis of voting algorithms.